Gustavo Diaz
Northwestern University
gustavo.diaz@northwestern.edu
gustavodiaz.org
Erin Rossiter
University of Notre Dame
erossite@nd.edu
erossiter.com
Paper and slides: gustavodiaz.org/talk
Explicit: Is some bias worth the increase in precision?
Implicit: Improving precision without sacrificing unbiasedness?
Explicit: Is some bias worth the increase in precision
Implicit: Improving precision without sacrificing unbiasedness?
Cost has to come from somewhere else!
Standard error of estimated ATE in conventional experimental design (Gerber and Green 2012, p. 57)
\[ SE(\widehat{ATE}) = \sqrt{\frac{\text{Var}(Y_i(0)) + \text{Var}(Y_i(1)) + 2\text{Cov}(Y_i(0), Y_i(1))}{N-1}} \]
\[ SE(\widehat{ATE}) = \sqrt{\frac{\text{Var}(Y_i(0)) + \text{Var}(Y_i(1)) + 2\text{Cov}(Y_i(0), Y_i(1))}{N-1}} \]
\[ SE(\widehat{ATE}) = \sqrt{\frac{\color{#4E2A84}{\text{Var}(Y_i(0)) + \text{Var}(Y_i(1)) + 2\text{Cov}(Y_i(0), Y_i(1))}}{N-1}} \]
Variance component
Decrease \(SE(\widehat{ATE})\) with alternative research designs
\[ SE(\widehat{ATE}) = \sqrt{\frac{\color{#4E2A84}{\text{Var}(Y_i(0)) + \text{Var}(Y_i(1)) + 2\text{Cov}(Y_i(0), Y_i(1))}}{N-1}} \]
Variance component
Decrease \(SE(\widehat{ATE})\) with alternative research designs
Block-randomization
Repeated measures
Pre-treatment covariates
Pair-matched design
Online balancing
Sequential blocking
Rerandomization
Matching
\[ SE(\widehat{ATE}) = \sqrt{\frac{\color{#4E2A84}{\text{Var}(Y_i(0)) + \text{Var}(Y_i(1)) + 2\text{Cov}(Y_i(0), Y_i(1))}}{N-1}} \]
Variance component
Decrease \(SE(\widehat{ATE})\) with alternative research designs
Block-randomization
Repeated measures
Pre-treatment covariates
Pair-matched design
Online balancing
Sequential blocking
Rerandomization
Matching
\[ SE(\widehat{ATE}) = \sqrt{\frac{\color{#4E2A84}{\text{Var}(Y_i(0)) + \text{Var}(Y_i(1)) + 2\text{Cov}(Y_i(0), Y_i(1))}}{N-1}} \]
Variance component
Decrease \(SE(\widehat{ATE})\) with alternative research designs
Block-randomization
Repeated measures
Pre-treatment covariates
Pair-matched design
Online balancing
Sequential blocking
Rerandomization
Matching
All require pre-treatment information
\[ SE(\widehat{ATE}) = \sqrt{\frac{\color{#4E2A84}{\text{Var}(Y_i(0)) + \text{Var}(Y_i(1)) + 2\text{Cov}(Y_i(0), Y_i(1))}}{N-1}} \]
Variance component
Decrease \(SE(\widehat{ATE})\) with alternative research designs
Block-randomization
Repeated measures
Pre-treatment covariates
Pair-matched design
Online balancing
Sequential blocking
Rerandomization
Matching
All require pre-treatment information
Two categories:
Reduce variance in observed outcomes
Reduce variance in potential outcomes
\[ SE(\widehat{ATE}) = \sqrt{\frac{\color{#4E2A84}{\text{Var}(Y_i(0)) + \text{Var}(Y_i(1)) + 2\text{Cov}(Y_i(0), Y_i(1))}}{\color{#00843D}{N-1}}} \]
Sample size component
\[ SE(\widehat{ATE}) = \sqrt{\frac{\color{#4E2A84}{\text{Var}(Y_i(0)) + \text{Var}(Y_i(1)) + 2\text{Cov}(Y_i(0), Y_i(1))}}{\color{#00843D}{N-1}}} \]
Sample size component
Quadruple to halve \(SE(\widehat{ATE})\)
Focus: Increasing numerator may come at the cost of decreasing denominator
Precision gains from alternative designs may be offset by sample loss